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Background / Context: 

Description of prior research and its intellectual context. 

In 2004, the New York City Department of Education (NYCDOE) instated a universal high 
school choice policy in which all incoming freshmen submit a list of up to 12 schools they would 
like to attend ranked in order of preference. A centralized matching system then assigns students 
to schools using an algorithm akin to that used to match medical school graduates to residency 
programs (Abdulkadiroglu, Pathak, & Roth, 2005; Bloom, Thompson, & Unterman, 2010). 
Roughly 80,000 students from 1,100 middle schools participate annually, choosing among 700 
programs at 360 sites. No student is permitted to attend a “default” school and avoid an active 
choice, and few are guaranteed their first choice. Rather, each student’s placement depends on 
admissions priorities, demand, and in some cases, schools’ own rankings of applicants. 

This new policy — the High School Application Processing System, or HSAPS — has many goals. 
One is to require that all students be engaged in the process of choosing their school. A second 
goal is to break the link between residential location and school assignment, by allowing students 
to apply to any school citywide. A third is to better facilitate matches between students’ interests 
and needs and they school they attend. Einally, by increasing competition, the policy aims to 
improve overall school quality. To date, we know little about how students have fared in this 
large and diverse system of school choice. However, excess demand for popular and top- 
performing schools results in many students not receiving one of their top choices. 

Prom a methodological standpoint, HSAPS offers an unprecedented opportunity to understand 
the impact of school choice on student outcomes. Many NYC high schools are oversubscribed, 
meaning that more students apply for admission than there are seats. In these cases, HSAPS 
either awards seats at random (for non-selective programs), or in order of the schools’ ranked 
preferences (for selective programs). This assignment mechanism generates hundreds of natural 
experiments that can be exploited to rigorously contrast the outcomes of otherwise equivalent 
students who were and were not admitted to their school of choice. 

This paper is part of a larger project investigating the student-level impacts and system-level 
effects of universal choice in NYC. As one step toward understanding the impact of choice on 
achievement, we examine whether admission to a preferred school impacts critical mediators of 
student success. Por example, we ask whether students are more engaged with school or have 
better attendance when successfully matched to their first choice versus their second (or lower) 
choice. As we describe below, an important component of the study will address heterogeneity in 
effects across student populations and school types. We focus here on the population of students 
subject to random assignment, thus providing a plausibly causal impact estimate of attending a 
more preferred high school. 

This study is one of the first to examine effects of school choice on mediators of student 
achievement. Moreover, it does so in a context of mandatory choice, in which all students must 
are required to choose a school. Existing evidence on the effects of school choice participation 
primarily comes from voluntary “opt out” programs, such as open enrollment, vouchers, charter. 
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or magnet schools (e.g., Cullen, Jacob, & Levitt, 2006; Deming, Hastings, Kane, & Staiger, 
2009; Hoxby & Murarka, 2009). This work typically finds a positive or null effect of attending a 
chosen school on academic and other outcomes, though there is considerable heterogeneity 
across subgroups. For example, Deming et al. (2009) found students zoned to the lowest- 
performing schools saw the greatest benefits from attending a preferred school. Others found 
greater returns for more advantaged or motivated students (Lai, 2007; Hastings et ah, 2009; 
Lauen, 2009). 

It is less clear why students benefit from high school choice, or why these returns vary across 
populations. In one study of behavioral effects of school choice, Cullen et al. (2005, 2006) found 
that Chicago students admitted via lottery to their first choice high school were less likely to be 
involved in disciplinary incidents or arrests than those who lost these lotteries. Similarly, 
Imberman (2011) found positive effects of charter school attendance on the behavior of middle 
and high school students. Our study contributes to this literature by expanding the set of 
outcomes to student-reported measures of engagement and attendance, and by estimating effects 
for a large population of students and diverse set of subgroups subject to mandatory choice. 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 

This paper estimates the impact of attending a preferred high school on mediating factors of 
student success, including engagement, behavior, and attendance. For example, we ask whether 
students are more engaged with their school or have higher attendance when successfully 
matched to their first choice versus their second (or lower) choice. By focusing on the subset of 
incoming freshman who are assigned at random to their preferred school by the HSAPS 
mechanism (cases of oversubscription), we provide a plausibly causal estimate of the impact of 
attending a preferred school on these factors. In addition to estimating the average impact across 
all students, we explore heterogeneity in treatment effects across student populations of interest 
(e.g. economically disadvantaged, ELL, and low, middle, and high-achieving students). 

Setting: 

Description of the research location. 

This study takes place in New York City, a setting with a large and diverse population of public 
school students, enabling us to examine the effects of choice for many interesting subgroups, 
many of whom may not have otherwise participated in voluntary school choice. The number of 
available school choices is also large and diverse, with close to 700 programs at 360 school sites. 
NYC’s public transportation system provides students accessibility to a great many of these, 
offering a large menu of options for its students. 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features, or characteristics. 

Our study relies on two cohorts of rising 8th grade students in New York City in 2007-08 and 
2008-09, approximately 150,000 students. As seen in Table I, which focuses on the 2007-08 
cohort, our study population reflects the size and diversity of the public schools in NYC. 
Roughly 32% of these students are black, 40% Hispanic, and 27% white or Asian. Close to 60% 
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are eligible for free or redueed priee lunch (a proxy for low income). Many students are English 
language learners (11%), and more than 6% are recent immigrants. On a baseline of 79,000 
students in this one cohort, all of these subgroups are sizable. 

Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and duration. 

HSAPS was implemented in 2004 as part of a larger set of education reforms under Mayor 
Michael Bloomberg and Chancellor Joel Klein. The new policy required all 8th grade students to 
submit an application listing up to 12 high school programs, ranked in order of preference. 
Selective schools rank applicants without knowledge of the students’ rankings. HSAPS’ 
matching mechanism then assigns students to programs based on admissions method (i.e., school 
type), eligibility, admissions priorities, seat availability, and the ranked preferences of students 
and schools (Abdulkadiroglu et ah, 2005; Bloom et ah, 2010, Corcoran & Levin, 2011). 

NYC high schools can be divided into one of six school types, or admissions methods, which 
determine how HSAPS assigns students to the school. For example, limited unscreened schools 
are non-selective, but give priority to students who live in a defined geographic area. On the 
other hand, screened schools rank students and HSAPS uses these rankings when assigning 
students to schools. The number of applicants and school type determines for whom the school is 
oversubscribed; a school can be oversubscribed for some priority groups but not others. 

When a non-selective school is oversubscribed, students are assigned randomly within priority 
group. For example, in the case of “limited unscreened” schools, students who live in the same 
borough as the school and attended an information session are given first priority. In the case of 
selective schools, seats are allocated in order of the school’s rankings. In all cases, HSAPS 
attempts to assign students to their highest ranked choice with seats available. In Table 2 we use 
HSAPS data to estimate the percentage of first choices made by students that were considered 
oversubscribed. For non-selective schools such as limited unscreened and educational option, we 
estimate that 53-63% of first choices were oversubscribed in 2007-08. 

Research Design: 

Description of the research design. 

As an impact analysis, our most substantial challenge lies in identifying appropriate comparison 
groups for students receiving their school of choice. In most settings, selection complicates 
identification: choices are systematically related to factors related to outcomes of interest. 
Fortunately, many NYC high schools are oversubscribed, in which case HSAPS awards seats 
randomly or by schools’ rankings, depending on the school type. Assuming randomization works 
as intended, students subject to random assignment should be equivalent along observable and 
unobservable dimensions, thus providing appropriate comparison groups. 

For the subset of students admitted at random to their nth school choice (e.g. 1st) by the HSAPS 
mechanism, we estimate the impact of attending this school relative to a less preferred choice, 
comparing outcomes for those who identically ranked the same nth choice. For example, for 
outcome Y for student i (engagement, suspension, or attendance), we estimate the regression: 
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(1) Yi- a Is + yZii + f^Xi + Ui 



where Zh=\ if student i is assigned to her first ehoiee sehool (whieh is random, eonditional on 
priority group), au is a first ehoiee sehool fixed effeet (effeetively eomparing students within 
“lotteries”), and Xi is a set of student-level eovariates. The parameter y is the average impact of 
attending a first choice school on the outcome Y. This model is estimated for the pooled sample, 
for each (non-selective) school type, and for subgroups of interest. We also extend this model to 
incorporate all students assigned to a non-selective school through random assignment, allowing 
y to vary depending on whether assignment was to a first, second, third, or lower choice. 

Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 

The study draws on a collection of administrative databases provided by the NYCDOE. The 
HSAPS database is the primary data source for student choices. In addition to all students’ 
ranked high school choices, this data includes individual and contextual variables, including age, 
feeder school, academic history (including middle school course transcripts), attendance record, 
achievement test scores, special education and limited English proficiency status, school 
rankings, and ultimate school placement. 

This data is supplemented with other student characteristics, attendance and suspension records 
in high school, as well as aggregate characteristics of middle and high schools from databases 
compiled by the Research Alliance for NYC Schools. Using unique identifiers, all student 
records are matched to student-level responses on the annual NYC School Survey. Because 
survey data is only available from 2008 forward, we rely exclusively on data from the 2007-08 
and 2008-09 cohorts. Erom the School Survey we focus on the NYCDOE’s Engagement scale 
(or~0.83), which includes items such as “I feel welcome in my school,” and “My school offers a 
wide enough variety of classes and activities to keep me interested in school,” and has a standard 
deviation of .17 on a 4-point scale. In these years, the student response rate to these surveys was 
72-74%. 



Findings / Results: 

Description of the main findings with specific details. 

This study is in progress. We have compiled all of the necessary data for the two cohorts under 
investigation, and analysis is currently underway. A descriptive analysis of student choices and 
placements (which did not include School Survey results, nor exploited the randomization 
mechanism) was published in Spring 2011 by Corcoran & Eevin (2011). 

Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 

Universal choice policies like the one adopted in NYC have the potential to dramatically alter the 
playing field of educational opportunity for urban youth. Our study will provide new evidence on 
the impact of universal choice on a large population of students and subgroups, and shed light on 
mechanism through which school choice affects students’ academic success. 



SREE Spring 2012 Conference Abstract Template 



4 




Appendices 

Not included in page count. 



Appendix A. References 

References are to he in APA version 6 format. 

Abdulkadiroglu, A., Pathak, P. A., & Roth, A. E. (2005). The New York City high sehool mateh. 

American Economic Review, 95{2), 364-367. doi;10. 1257/000282805774670167 

Bloom, H. S., Thompson, S. L., & Unterman, R. (2010). Transforming the high school 

experience: How New York City's new small schools are boosting student achievement and 
graduation rates. New York: MDRC. Retrieved from 
http://www.mdre.org/publieations/560/full.pdf 

Coreoran, S. P., & Levin, H. M. (2011). School choice and competition in the New York City 
schools. In J. A. O'Day, C. S. Bitter & L. M. Gomez (Eds.), Education reform in New York 
City: Ambitious change in the nation's most complex school system (pp. 199-224). 
Cambridge, MA: Harvard Education Press. 

Cullen, J. B., Jacob, B. A., & Levitt, S. (2005). The impact of school choice on student 

outcomes: An analysis of the Chicago Public Schools. Journal of Public Economics, 59(5- 
6), 729-760. doi:10.1016/j.jpubeco.2004.05.001 

Cullen, J. B., Jacob, B. A., & Levitt, S. (2006). The effect of school choice on participants: 
Evidence from randomized lotteries. Econometn'ca, 74(5), 1191-1230. doi:10.1 1 1 l/j.l468- 
0262.2006.00702.x 

Deming, D., Hastings, J., Kane, T., & Staiger, D. (2009). School choice and college attendance: 
Evidence from randomized lotteries. Unpublished manuscript. Retrieved from 
http://aida.econ.yale.edu/~ih529/papers/DHKS CollegeGoing DraftSept09.pdf 

Hastings, J. S., Kane, T. J., & Staiger, D. O. (2009). Heterogeneous preferences and the efficacy 
of public school choice. Unpublished manuscript. Retrieved from 
http://www.econ.yale.edu/~ih529/papers/HKS Combined 200806.pdf 

Hoxby, C. M., & Murarka, S. (2009). Charter schools in New York City: Who enrolls and how 
they affect their students' achievement (NBER Working Paper No. 14852). Cambridge, MA: 
National Bureau of Economic Research. Retrieved from 
http://www.nber.org/papers/wl4852 

Imberman, S. A. (2011). “The effect of charter schools on achievement and behavior of public 
school students.” Jowma/ of Public Economics 95(7-8):850-863. 

Lai, F. (2007). The effect of winning a first-choice school entry lottery on student performance: 
Using a natural experiment from the middle school reforms in Beijing. New York: National 
Center for the Study of Privatization in Education at Teachers College, Columbia 
University. Retrieved from: http://www.ncspe.org/publications files/OP139.pdf 



SREE Spring 2012 Conference Abstract Template 



A-1 



Lauen, D. L. (2009). To choose or not to choose: High school choice and graduation in Chicago. 
Educational Evaluation and Policy Analysis, 31(3), 179-199. 



doi: 10.3 102/0162373709339058 



SREE Spring 2012 Conference Abstract Template 



A-2 




Appendix B. Tables and Figures 

Not included in page count. 

Table 1: Characteristics of 8th grade students participating in HSAPS, 2007-08 cohort 



Student characteristic 


(%) 


Black 


32.2 


Hispanic 


39.8 


Asian 


13.9 


White 


13.6 


Female 


49.0 


Free lunch eligible 


59.8 


Recent immigrant 


6.3 


ELL 


11.1 


Overage (15+) 


9.7 


High reading level 


16.8 


Low reading level 


12.2 


Middle reading level 


70.9 


Borough: 




Manhattan 


12.0 


Brooklyn 


31.5 


Queens 


27.5 


Bronx 


22.8 


Staten Island 


6.2 


N 


78,991 
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Table 2: Estimating the number of students alloeated to their first choice sehool through random or rank-based assignment: 2007-2008 



Program type 


(A) 

Number 
ranking 
program first 


Col. % 


(B) 

Of (A), 
received 
first choice 


%of 

(A) 


(C) 

Of (A), estimated 
choices that were 
oversubscribed 


%of 

(A) 


(D) 

Of(C), % 
getting 
first choice 


Zoned 


5,925 


8.9 


4,141 


69.9 


2,333 


39.4 


27.5 


Unscreened 


1,272 


1.9 


863 


67.8 


516 


40.6 


25.2 


Limited unscreened 


10,928 


16.5 


8,069 


73.8 


5,818 


53.2 


56.7 


Screened 


23,132 


34.8 


8,484 


36.7 


5,071 


** 21.9 


69.5 


Audition 


4,858 


7.3 


1,719 


35.4 


350 


*** y 2 


58.9 


Educational option 


20,261 


30.5 


12,754 


62.9 


12,872 


63.5 


47.6 


Total (or % of all) 


66,376 


- 


36,030 


54.3 


26,960 


40.6 


- 



Notes : eolumn (A) ineludes applieants from publie and private sehools who did not aeeept a speeialized high sehool offer if given one, 
and were not ineligible for their first ehoiee, and were ultimately finalized to some program (a missing final program may indieate that 
the student left the distriet or withdrew from the proeess). Whether or not a first ehoiee is determined oversubscribed depends on the 
sehool type (i.e., admissions method) and the priority group that the student is a member of (see details below). 

Zoned : a first ehoiee is oversubseribed: (1) only for students who reside outside of the residential zone, and (2) when there are more 
out-of-zone first ehoiee applieants than first ehoiees awarded (we require a minimum of 5 more applieants than seats awarded). 

Unsereened and limited unsereened : a first ehoiee is oversubseribed if there are more first ehoiee applieants within one’s priority 
group (e.g. geographie preferenee eategory) than there are first ehoiees awarded to this priority group (we require a minimum of 5 
more applieants than seats awarded, and a minimum of 1 seat awarded; groups where no student reeeived their first ehoiee were not 
oversubseribed). 

Sereened : a first ehoiee is oversubseribed if the student has been ranked by the school and there are more school ranked first ehoiee 
applieants than there are school ranked first ehoiees awarded (we require a minimum of 5 more applieants than seats awarded). 
**21.9% represents the pereent of all sereened program first ehoiees that were oversubseribed. This appears low beeause in order to be 
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considered oversubseribed, the school had to have ranked the student (only 43.3% are ranked). Of those ranked by the school, 50.5% 
would be eonsidered oversubscribed choices. 



Audition : same rules as screened. Note that only 1,896 (39.0%) of the 4,858 audition program applieants were ranked at all by the 
school. The remaining 61.0% are never in the running for their first choice. (Although a small share, 0.9%, of the first choice 
applieants who were not ranked by the sehool ended up getting their first ehoice). represents the percent of all audition 

program first ehoices that were oversubscribed. This appears low because in order to be eonsidered oversubscribed, the school had to 
have ranked the student (and as note, only 39.0% are ranked). Of those ranked by the school, 18.5% would be eonsidered 
oversubscribed ehoices. 

Educational option : a first choice is oversubscribed if there are more first ehoice applicants within one’s priority group and reading 
ability level (low, middle, high) than there are first choices awarded to this priority group and reading ability level (we require a 
minimum of 5 more applieants than seats awarded to this group). 
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